Using PCA to Efficiently Represent State Spaces

نویسندگان

  • William Curran
  • Tim Brys
  • Matthew E. Taylor
  • William D. Smart
چکیده

Reinforcement learning algorithms need to deal with the exponential growth of states and actions when exploring optimal control in high-dimensional spaces. This is known as the curse of dimensionality. By projecting the agent’s state onto a low-dimensional manifold, we can represent the state space in a smaller and more efficient representation. By using this representation during learning, the agent can converge to a good policy much faster. We test this approach in the Mario Benchmarking Domain. When using dimensionality reduction in Mario, learning converges much faster to a good policy. But, there is a critical convergence-performance trade-off. By projecting onto a low-dimensional manifold, we are ignoring important data. In this paper, we explore this trade-off of convergence and performance. We find that learning in as few as 4 dimensions (instead of 9), we can improve performance past learning in the full dimensional space at a faster convergence rate.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Application of Architectural Visual Documents and Oral History of in the Representation of Micro-Spaces and Three-Dimensional (3D) Modeling of Nawab Razavi Historical House in Yazd

Over the time, various factors have led to damage the Iranian houses. By examining the surviving documents of Nawab Razavichr('39')s house in Yazd, it is possible to represent a major part of the lost spaces and also to minimize speculation in the restoration of this historic house. The basic belief of this research is that the studies of the oral history of architecture as well as the existing...

متن کامل

Representing Spectral data using LabPQR color space in comparison to PCA method

In many applications of color technology such as spectral color reproduction it is of interest to represent the spectral data with lower dimensions than spectral space’s dimensions. It is more than half of a century that Principal Component Analysis PCA method has been applied to find the number of independent basis vectors of spectral dataset and representing spectral reflectance with lower di...

متن کامل

Generalized N-dimensional principal component analysis (GND-PCA) and its application on construction of statistical appearance models for medical volumes with fewer samples

We propose a method called generalized N-dimensional principal component analysis (GND-PCA) for the modeling of a series of multi-dimensional data in this paper. In this method, the data are directly trained as the higher-order tensor and the bases in each mode subspace are calculated to compactly represent the data. Since GND-PCA analyzes the multi-dimensional data directly on each mode better...

متن کامل

Distributed Symbolic Reachability Analysis

Model checking is an important tool in program verification and software validation. Model checkers generally examine the entire state space of a model to find behaviour that differs from a given formal specification. Most temporal safety properties can be verified via reachability analysis. A major limitation is the state space explosion problem, which occurs when the state space does not fit ...

متن کامل

Robust state estimation in power systems using pre-filtering measurement data

State estimation is the foundation of any control and decision making in power networks. The first requirement for a secure network is a precise and safe state estimator in order to make decisions based on accurate knowledge of the network status. This paper introduces a new estimator which is able to detect bad data with few calculations without need for repetitions and estimation residual cal...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • CoRR

دوره abs/1505.00322  شماره 

صفحات  -

تاریخ انتشار 2015